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A METHOD AND SYSTEM FOR VOICE ON 
DEMAND PRIVATE MESSAGE CHAT 

BACKGROUND OF THE INVENTION 
Field of the Invention 

[0001] The present invention relates generally to systems and methods of non-synchronous 
communication and, particularly to a system and method for voice on demand elements of 
private message chats, where end users may elect the modality of their chat communication for 
each receipt of a message, and each response. 

Description of the Prior Art 

[0002] Traditional telephony allows a variety of calling paradigms including: one to one (regular 
call), one to many (party line on receiving side), and many to many (conference call). Each 
conversation traditionally requires a circuit, and the call flow follows the path of call request 
(dial), call setup, call accept (on the receiver side), cut through and conversation. When one 
party hangs up the call is terminated. Call setup often involves a long time, up to 30 seconds, 
and the nature of the telephone switch prevents rapid call termination/initiation. When one party 
hangs up and then attempts to reinitiate a call to the same number, the line is often still busy, and 
it is necessary to wait until it clears before completing the call. 

[0003] Voice over Internet Protocol (VOIP) technologies provide packet based voice 
replacements for traditional telephony. Many novel approaches are possible to initiate calls, 
such as click-to-call on web sites (used for help desk coverage, among other things), VOIP can 
provide the same functions - conferencing, long distance, etc. as traditional voice telephony. 

[0004] In conventional Internet "chat" environments, text conversations via computing devices 
are held in a semi-synchronous fashion. The chat environment is a chat room, e.g., a Web site, 
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part of a Web site, or part of an online service such as America Online, that provides a venue for 
users with a common interest to communicate in real time. Unlike forums and discussion 
groups, chat services have the capacity for interactive messaging and do not require users to have 
any special software. Internet Relay Chat, which is a system for chatting that involves a set of 
rules and conventions, does require client/server software, which is capable of being downloaded 
from the Internet. Chat room users register for a chat room, choose a user identification and 
password, and log into that particular room. Inside the chat room, generally there is a list of the 
people currently online, each of whom also are alerted that another person has entered the chat 
room. To chat, users type a message into a text box and the message is virtually immediately 
rendered visible in a larger display area so that other users may now respond. 

[0005] Figure 1 illustrates a conventional chat system 100. In the chat system 100 depicted in 
Figure 1, elements 110 and 140 represent chat client devices physically embodied as a server, a 
personal computer (PC), a personal digital assistant (PDA), a cellphone or other like computing 
device. Examples of capable non-personal computer devices include Blackberry devices 
(available at www.blackberry.com) and AOL which provides Instant Messaging (IM) service for 
cellphones (http://mymobile.aol.com/portal/im/index.html). Elements 110 and 140 need not 
represent equivalent devices (e.g., need not both be personal computers), as long as each is 
capable of providing the infrastructure of a chat client. Generally, devices must have the ability 
to accept chat input, and to display chat output. Software providing the chat function for devices 
1 10 and 140 may be resident locally, or may be provided as a service by a remote server (not 
shown). Remote servers providing such software may be different for different clients (e.g., 
devices 1 10 and 140 may be served by different remote software servers). As shown in Figure 1 , 
chats in progress result in text transcripts 120 and 150 that generally provide an indicator of the 
source of a chat statement (e.g., a nickname for whomever entered the statement), and a textual 
transcript of the chat statement itself. These are often provided in a window that maintains the 
sequential nature of the interchange, and provides the visual frame associating all the messages. 
Elements 1 10, 140 and a network-based chat service 180 are shown interconnected via a 
communications network. 170 which may comprise a single network, or a set of interconnected 
networks including links which may be wireless, wireline, broadband and narrowband, and may 
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include the Internet in general. Chat system 100 additionally include elements 130 and 160 that 
let chat clients speak to one another via network 1 70 in a manner much like the experience 
provided by a telephone call. Elements 130 and 160 particularly enable a client to speak to the 
other chat client, and hear what the other chat client has said. An example of an audio-capable 
client device is a personal computer with a microphone and speaker either built in or added on. 
In addition to the basic textual chat system described above, an audio-capable system requires 
software for interfacing to the speaker and microphone, encoding and decoding the audio data, a 
protocol for exchanging the audio data between client devices, and a network 170 with sufficient 
capacity and bandwidth to send and receive audio data at a rate consistent will spoken 
conversation. It should be understood that conventional chat systems treat audio and textual data 
differently. While transcripts 120 and 150 provide a chronological history of the textual 
messages exchanged in the chat session, they do not include audio data. Audio data is sent from 
the originating client, and played at the receiving client as it would during a telephone call that 
was not being recorded. 

[0006] Through chat embodiments such as Lotus Sametime (an instant messaging and Web 
conferencing solution for businesses) and AOL Instant Message, individuals may engage in 
lightweight text interchanges with: 1) minimal overhead to start, 2) minimal expectations as to 
length of message, persistence, acknowledgement, 3) minimal requirements for rapid response 
on the part of the recipient. Chat is a conversation, but not a real time conversation (i.e., not 
synchronous). The sender expects a response, but not necessarily right away. However, unlike 
e-mail and voicemail, there's an expected thread of continuity between interchanges, presented 
visually with the ability to display the whole of the interchanges within one chat window per 
conversation. Such chats may generally be held concurrently, allowing an individual to 
communicate with many people at once on different topics. 

[0007] Current e-mail systems now enable the addition of voice elements. For example, audio 
file formats such as *.wav which are digital audio captured in a file, may now be appended as an 
attachment to e-mail. It is well known in the art that one can attach audio files to e-mail 
messages, which may subsequently be played by the recipient at his location. A unified 
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messaging system (e.g., see http://www.iec.org/odine/tutorials/iinified_mess/ ) uses this kind of 
communication to move voice mails around in a fully asynchronous manner. The "Notes 
Buddy" tool for Lotus Notes™ is an exemplary embodiment that integrates instant messaging 
(chats) and buddy status with e-mail to produce a single messaging tool. With notes buddy, one 
can use voice to input to a Notes e-mail which is sent to recipient and appears as a *.wav file 
attachment to a regular email. Notes buddy also allows a user to play out his/her regular e-mail 
via text to speech conversion. It is noted that this program provides Lotus Sametime status in 
conjunction with the mail. Generally, however, an e-mail is sent and received without 
knowledge of the current availability or willingness of the other party to attend to the e-mail. 

[0008] AOL, MSN and Lotus Sametime provide the ability to conduct synchronous voice 
communications. These are fully synchronous "open mike" applications and do not allow any 
voice exchange that is not synchronous. These applications provide two parallel 
communications methods - one for text chat and one for synchronous voice. They do not 
provide semi-synchronous voice, nor the ability to store the voice utterances. 

[0009] Thus, in sum, today's telephone communication paradigm requires a connected circuit, a 
call-flow path of call request (dial), call setup, call accept (on the receiver side), cut through and 
conversation. When one party hangs up the call is terminated. While VOIP does not require a 
circuit, calls made with VOIP follow the same stringent paradigm, and imply a dedicated 
listener. While VOIP is now generally used as a replacement for voice telephony, it would be 
highly desirable to enable means for voice communications that does not require a dedicated 
listener. Further, while e-mail communications permit audio file attachments enabling 
asynchronous transmission of voice, it does not allow conversational use of voice on demand. It 
would thus be highly desirable to provide a means for adding voice on demand elements to data 
communication systems. 

[0010] As text chat is very much a hands on, eyes on communication method that does not allow 
for the ready capture of nuance as does voice. It would further be desirable to provide a "hands 
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off 5 , "eyes off version of chat that includes the provision of means for capturing and 
transmitting nuance as enabled with voice communications. 

SUMMARY OF THE INVENTION 

[00111 A method and system of the invention enables semi-synchronous voice, for applications 
such as chat. By breaking away from the traditional telephony call paradigm, a user of the 
system and method of the invention is enabled to speak to a device with an active chat window, 
and expect that the recipient of the chat will be able to hear the voice chat semi-synchronously. 
The recipient may then respond in a manner of his/her choice, including text chat, voice 
conversation, or, no response. The invention enables an individual to have conversation that 
ranges through a spectrum from fully synchronous to fully asynchronous, with the ability to alter 
the mode at any time. This conversation may be with one or more other individuals or may be 
with an IT process that uses TTS (text-to-speech)/voice recognition as the output/input 
mechanisms. 

[0012] In the present invention, a user (first party) of Voice On Demand For Private Message 
Chat (V ODPMC), may speak to a device with an active chat window, and expect that the 
recipient of the chat will be able to hear the voice chat semi-synchronously. The recipient then 
may respond in a manner of his or her choice, including text chat, VODPMC, voice 
conversation, or no response. Note that the first party user does not have the ability to eavesdrop 
on the recipient, and in fact may not expect to hear anything at the time of the message. 

[0013] The present invention further enables an individual to have a conversation that ranges 
through the spectrum from fully synchronous to fully asynchronous, from text to voice on 
demand, with the ability to alter the mode at any time. This conversation may be with one or 
more other individuals or may be with an IT process that uses TTS/voice recognition as the 
output/input mechanisms. 
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[0014] According to the invention, there is provided a system and method for establishing a 
private message chat between electronic devices. The method includes steps of providing an 
indication as to the availability of a user for receiving a private message chat; receiving an audio 
input message from at least one first client; and transmitting the audio input message to at least 
one second client over a communications network, wherein the at least one second client can 
receive, temporarily store and play back the audio input message. The first client may receive a 
reply audio input message from the at least one second client or, receive a reply text input 
message from the at least one second client, and subsequently may further communicate to the 
second client device via one of audio or text messaging, according to that user's discretion. The 
transmitting of any audio input message may be over a wired connection (Internet) or a wireless 
connection. 

[0015] Advantageously, the system for voice chat according to the invention may employ 
biometric devices to order to authenticate or provide a confidence factor that the person 
identified as communicating with a user of client device, is in fact the person identified. 

BRIEF DESCRIPTION OF THE DRAWINGS 

[0016] Further features, aspects and advantages of the structures and methods of the present 
invention will become better understood with regard to the following description, appended 
claims, and accompanying drawings where: 

[0017] Figure 1 depicts a conventional chat system; 

[0018] Figure 2 depicts the VODPMC chat system 200 of the invention; 

[0019] Figure 3 is a flow chart depicting the methodology for implementing VODPMC of the 
invention; and, 
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[0020] Figures 4(a)-4(c) depict an exemplary user interface for implementing VODPMC 
according to the invention. 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT 

[0021] The present invention is directed to a semi-synchronous voice chat system. Referred to 
.herein as Voice On Demand for Private Message Chat (VODPMC), the system and method 
provides a break from the traditional telephony call paradigm. 

[0022] Figure 2 depicts the VODPMC chat system 200 of the invention. In the inventive system 
200, elements 210 and 240 represent chat client devices within chat system 200, and element 280 
provides the network-based chat service. As in conventional chat systems, elements 210 and 240 
may physically represent a server, a personal computer (PC), a personal digital assistant (PDA), 
a cellphone or other computing device. Examples of capable non-personal computer devices 
include Blackberry devices (www.blackberry.com). AOL provides an Instant Messaging (IM) 
service for cellphones (http://mymobile.aol.com/portal/im/index.html). Devices 210 and 240 
need not represent equivalent devices (e.g., need not both be personal computers), as long as 
each is capable of providing the infrastructure for a chat client. Software providing the chat 
function for elements 210 and 240 may be resident locally, or may be provided as a service by a 
remote server (not shown). Remote servers providing such software may be different for 
different clients (e.g., elements 210 and 240 maybe served by different remote software servers). 
Element 280 may be a single server machine, a static or dynamic collection of machines 
performing the server function, or may be co-located on one or both of the machines serving as 
chat clients. Elements 210, 240, 280 are adapted for connection to each other via a 
communications network 270 which may comprise a single network, or a set of interconnected 
networks including links which are wireless, wireline, broadband and narrowband, and may 
include the Internet, in general. 

[0023] The chat system 200 of the invention additionally provides elements that enable chat 
clients to speak to one another in the style of communication provided by text-only IM systems. 
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The elements that provide the basic audio input and output capability are elements 230 and 260, 
and they allow a client to store and send an audio message to the other chat client, and for either 
chat client to playback audio messages. An example of an audio-capable client device is a 
personal computer with a microphone and speaker either built in or added on. Such an audio- 
capable client device includes software for interfacing to the speaker and microphone, encoding 
and decoding the audio data, for detecting the start and end of an audio message, and a protocol 
for exchanging the audio data between clients and, additionally includes a device to store and 
replay the audio data, and interface with a network 270. It is understood that the capacity and 
bandwidth of network 270 is not required to enable the sending and receipt of the audio data at a 
rate consistent with real-time spoken conversation. As the audio data is stored and replayed at 
both clients, the network 270 may deliver the audio at any rate that can be supported. 

[0024] It is understood that audio transmission may be accomplished using Voice Over IP 
(VOIP), IP telephony, or SIP (Session Initiation Protocol) which is a signaling protocol for 
Internet conferencing, telephony, presence, events notification and instant messaging. SIP 
particularly is an Internet Engineering Task Force (IETF) standard protocol for initiating an 
interactive user session that involves multimedia elements such as video, voice, chat, gaming, 
and virtual reality. It particularly provides the necessary protocol mechanisms so that end 
systems and proxy servers can provide voice telephony services including, but not limited to: call 
forwarding, callee and calling "number" delivery, where numbers can be any (preferably unique) 
naming scheme; caller and callee authentication; blind and supervised call transfer; invitations to 
multicast conferences; and, extensions of SIP to allow third-party signaling. 

[0025] The chat system 200 of the invention additionally enables all textual and audio messages 
to be saved in chat transcripts 220 and 250. Either chat client 210, 240 may re-read or re-play 
any message on demand. Such transcripts generally provide an indicator of the source of a chat 
statement (e.g., a nickname for whoever entered the statement), and, a textual transcript of the 
chat statement itself. For audio messages, voice recognition technology may be provided to 
generate an approximation of the audio content, or an iconic representation of the audio message 
which may be displayed along with information about the audio message (for example, the 
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duration of the message). These transcripts are provided in a window or similar interface that 
maintains the sequential nature of the interchange, and provides the visual frame associating all 
the messages. 

[0026] It is understood that the system 200 of the invention may be used even when element 230 
or 260 (but not both) are absent from the system. Those skilled in the art will recognize that 
speech recognition technology may be used to map audio messages into text, and that text to 
speech technology may be used to map text messages to audio. 

[0027] The system 200 may further be used in systems where one or both clients lacks the ability 
to enter textual messages, but has the ability to provide for chat transcripts 220 or 250 as 
described hereinabove. Examples of such chat clients include small mobile devices with ample 
capacity for audio data, but limited space for textual input entry and reading textual output. 
Current cell phone devices provide this capability, and it is expected that future cell phones will 
as well. 

[0028] Figure 3 is a flow chart depicting the methodology for implementing the VODPMC 
system of the invention. In a first step 310, an indication is generated for display on a client 
device to represent user availability for a private message chat. Such an indication may include, 
but is not limited to: an indication of presence, an indication of availability, an e-card providing a 
channel for a private message chat; a text element, a graphic element, an icon, a color, or a 
texture. Existing systems adapted to provide such indications include Lotus Sametime 
(presence), and AOL IM. An "e-card", such as described in commonly-owned, co-pending U.S. 
Patent Application Serial No. 036,194 entitled SYSTEM AND METHOD FOR PROVIDING 
ACCESS AND UTILIZATION OF CONTEXT INFORMATION, and in commonly-owned, co- 
pending U.S. Patent Application Serial No. 10/624,107 entitled COMMUNICATION 
CAPABILITY COUPONS is adapted to display multiple channels that are available for 
communications and which among these channels are available at a current time. An indication 
of availability may include, but is not limited to: indications of willingness, indications of 
preference for communication, or indications of calendar availability. The whole contents and 
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disclosures of commonly-owned, co-pending U.S. Patent Application Serial Nos. 036,194 and 
10/624,107 are incorporated by reference as if fully set forth herein. At the completion of step 
310, a potential user of voice on demand private message chat is aware that a second user is 
available. Thus, the sender has knowledge via this availability indicator that the recipient is 
available to receive a semi-synchronous voice chat message now and may expect an imminent 
response. It should be understood that certain client devices may be utilized by users who cannot 
use their hands to communicate via client devices of the VODPMC system. An example of such 
a user is a driver of an automobile who must communicate by "hands free" communications 
means. Thus, in an alternate embodiment, an audible indication may be provided to a user to 
indicate the availability of a second user for receiving voice on-demand messages. 

[0029] Proceeding next to step 320, Figure 3, there is depicted a step of receiving audio input 
from a client. The client may receive the audio through a microphone, through a network 
connection, or other means. For instance, a wireless network implementing Bluetooth or like 
wireless communications protocol from a wireless connected peripheral may be employed. 

[0030] As depicted at next step 330, audio is transmitted to at least a second user for storage 
thereat. This audio may be transmitted directly to the second chat client, by wireless or wired 
network connection, or may be transmitted through an intermediate network such as the Internet. 
It is understood that, alternately, the audio input may be stored at the chat server 280 (Figure 2) 
until accessed by the second chat client. Regardless, this audio may or may not traverse an 
intermediate application or chat processor. If the audio traverses an intermediate processor, then 
the intermediate processor may provide additional function such as conferencing, language 
translation, audio transformation, transcription or other service. 

[0031] Next, at step 340, Figure 3, there is depicted the step of determining whether the user is 
able to receive a private message chat. Despite the indication of availability in step 310, it may 
be that the at least one second user cannot receive a chat. The second user may have inadequate 
hardware, may have experienced reception difficulty due to wireless or wired network outage, or 
may have signaled a subsequent unavailability to receive the private message chat. If the 
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decision is that the user cannot receive a private message chat, then the process proceeds to step 
370 to perform error processing. Following step 370, the process ends at step 380. 

[0032] Returning to step 340, if it is determined that the user could receive a private message 
chat, the process proceeds to step 350, where an indication is provided to the at least one second 
user of the audio element of the private message chat, i.e., a voice on demand element. This 
indication may include but is not limited to: an audio indication such as a bell, a visual indication 
such as a blinking light, a text indication, a graphic indication, a color indication, a texture 
indication, or a displayed icon. 

[0033] Next, at step 360, the audio element is rendered responsive to an indication from the at 
least one second user. Such indication may include but is not limited to: a preference, a profile, a 
keystroke, a voice command, a touch screen action, a button press, a motion, or a gesture. A 
voice on demand private message chat is now established, and the second client user may now 
respond by voice and/or text. The process then terminates at step 380. It should be understood 
that in an example of a "Hands free" chat session, the second user may render response messages 
as audio input for receipt by a first client device for receipt by a driver of an automobile. 

[0034] Illustrative examples of the VODPMC of the invention are now provided. In each 
example, close collaboration between two parties is enabled using a chat application capable of 
VODPMC in addition to textual messages and fully synchronous voice communication, and 
which provides a user interface indicating communication channel(s) available from one party to 
another, along with availability. 

[0035] In a first example, a VODPMC one-way session is established between two parties that 
includes an initial audio communication, e.g., of a user's voice. The VODPMC-capable 
application provides an interface and enables Joe to speak without further action, but does not 
allow him to listen. Jane likes to work at the board, and often does not pay attention to her PC. 
She ignores all the IM beeps. Joe has an important question, so he clicks on Jane ? s e-card to 
establish the one-way session and says "Jane-can you talk for a minute?". The voice is 
transmitted to Jane's (client) device via a wired connection, e.g., VOIP, or a wireless connection, 
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e.g., RF communication, rendered as audio, and stored as an audio file or as an in-memory 
temporal audio object for subsequent replay at Jane's (client) device. If Jane is in the room, and 
has enabled VODPMC sessions from Joe to be rendered (played out loud) as soon as they arrive 
at her client, she hears Joe, and can click a button or icon enabling fully synchronous voice 
communication- this action will permit Joe to hear as well as speak, and they converse. Thus, in 
this first example, the inventive system and method establishes a VODPMC one-way session 
with Jane's system, allowing him to speak effortlessly to her in response to Joe ! s click of her e- 
card. At first, Joe cannot hear Jane (to ensure her privacy) without a specific action by her 
allowing it. It is understood that, in this example, the verbalization is logged and a visual 
indication is provided on the VODPMC-capable application interface that something new is 
there. If Jane was out of the room, or had not enabled the rendering of audio upon receipt, then 
she eventually comes back, and/or notices that something new is there, clicks on the 
communication request and the logged verbalization is played back. In this case, the audio is 
VODPMC. 

[0036] A second example implementing the system and method of the invention is depicted in 
Figures 4(a)-4(c). In this example of VODPMC 400 shown in Figure 4(a), a session is 
established between two parties in which the second party does not hear the initial audio message 
at the time it's received. In a first step, Bob's VODPMC-capable chat interface 410 provided at 
his client device indicates that Jim is presently on-line and available for a VODPMC request. At 
step 412 Bob clicks on Jim ! s name (or an iconic representation thereof), and says "We're in the 
cafeteria". The voice is transmitted to Jim's (client) device via a wired connection, e.g., VOIP, 
or a wireless connection, e.g., RF communication, and the verbalization is rendered as audio and 
stored as an audio file (or, as an in-memory audio object) for subsequent replay via Jim's (client) 
device. As shown in Figure 4(b), Jim, via his VODPMC-capable chat interface 420 provided via 
his client device, sees an indicator 425 that there's an audio element to the chat message. 
Optionally, a version of what Bob had recited is transcribed through a voice recognition unit at 
Jim's client device or the network chat server, and additionally indicated as text message 426 via 
the interface 420. Depending upon Jim's circumstance or preference, he may choose to listen to 
the audio, (e.g., if he is not in a customer meeting), or, can simply use the voice recognized 
transcribed version. Jim now can decide how to respond - either with synchronous voice, a 
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VODPMC, or with text. The example depicted in Figure 4(c) indicates a selection 430 via Jim's 
interface for enabling a VODPMC or synchronous voice response back to Bob. 

[0037] In these examples, when these VODPMC-capable chat (interface) windows are closed, 
the chat, including the indications of audio elements, disappears. If the recipients did not listen 
to the audio elements, they no longer have the option to do so. 

[0038] It should be understood that a "Hands free" VODPMC chat session may be established 
according to the invention whereby one party via a first client device communicates by voice and 
the second client responds by text, using text to speech (TTS) for rendering. This may be 
advantageous for a driver implementing a first client device who is unable to chat via text 
messaging. This client (a driver in an automobile) may first indicate his availability to receive 
synchronous voice or VODPMC. Subsequently, another party desirous of communicating with 
the driver may initiate VODPMC communications, or may respond by text messaging, if voice is 
not available. In the instance voice response is not available, the driver's client device or 
network chart server will employ a TTS feature for receiving and playing voice messages from 
any text message communicated by the other party. 

[0039] It should be further understood that the system for voice chat according to the invention 
may employ biometric devices in order to authenticate or provide a confidence factor that the 
person identified as communicating with a user of client device, is in fact the person identified. 
As known to skilled artisans, more sophisticated biometric analysis systems may be 
incorporated, for instance, to ascertain emotional status of a communicating party, e.g., via 
employed heart-rate sensors, voice tonal quality analysis, keyboard force sensors, etc. 

[0040] It is further understood that the system for voice chat according to the invention may be 
oriented for multiple modes of use, i.e., a user (recipient) may employ two client devices: e.g., a 
Personal Digital Assistant (PDA) and a cell phone. In such implementation, a user may receive 
audio on the first device (e.g., cell phone), yet reply via his/her PDA. 
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[0041] Further to this, it is understood that a user, in a single chat session, may employ both 
voice and text in communications. That is, the use of voice and text may be interleaved 
throughout a single chat session, with the user, for example, using text response messaging or 
voice response, depending upon the communication circumstances, e.g. user environment. 

[0042] While the invention has been particularly shown and described with respect to illustrative 
and preferred embodiments thereof, it will be understood by those skilled in the art that the 
foregoing and other changes in form and details may be made therein without departing from the 
spirit and scope of the invention that should be limited only by the scope of the appended claims. 
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